Displaying modified stereo visual content

ABSTRACT

Examples are disclosed that relate to devices and methods for displaying stereo visual content via a head-mounted display (HMD) device. In one example, a method comprises: establishing a default display distance from an origin in a virtual coordinate system; determining a modified display distance from the origin; determining that visual content comprises stereo visual content comprising a left eye image and a right eye image; based on determining that the visual content comprises stereo visual content, scaling the left eye image to a scaled left eye image and scaling the right eye image to a scaled right eye image using a scaling factor that is proportional to a difference between the modified display distance and the default display distance; and displaying the scaled left eye image and the scaled right eye image at the modified display distance.

BACKGROUND

Stereoscopic displays can introduce distortions and/or spatialmisperceptions of displayed content based on viewer movement. Ahead-mounted display device also may display stereoscopic contentoriginally produced for traditional fixed-position displays.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

Examples are disclosed that relate to devices and methods for displayingvisual content via a head-mounted display (HMD) device. In one example,a method comprises: establishing a default display distance from anorigin in a virtual coordinate system, setting a modified displaydistance from the origin, determining that the visual content comprisesstereo visual content comprising a left eye image and a right eye image,based on determining that the visual content comprises stereo visualcontent, scaling the left eye image to a scaled left eye image andscaling the right eye image to a scaled right eye image using a scalingfactor that is proportional to a difference between the modified displaydistance and the default display distance, and displaying the scaledleft eye image and the scaled right eye image at the modified displaydistance.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram illustrating an example system fordisplaying visual content via an HMD device according to examples of thepresent disclosure.

FIG. 2 is an illustrative example of a use case scenario in which a userviews stereo visual content and three-dimensional content at a defaultdisplay distance according to examples of the present disclosure.

FIG. 3A shows a simplified top-down view of the room of FIG. 2.

FIG. 3B shows a simplified top-down view of the room of FIG. 2 in whichthe stereo visual content is displayed at a modified display distance.

FIGS. 4A and 4B show an illustrative example of scaling and displayingstereo visual content at a modified display distance according toexamples of the present disclosure.

FIG. 5 shows an example of setting the modified display distance to adistance from the origin to a planar surface according to examples ofthe present disclosure.

FIG. 6 shows an example of an occluding object between a location of theHMD device and a location of the visual content at the modified displaydistance according to examples of the present disclosure.

FIG. 7 shows an example of shortening the modified display distance to ashortened modified display distance according to examples of the presentdisclosure.

FIGS. 8A and 8B show illustrative examples of changing a displayedposition of a right eye image and/or left eye image according toexamples of the present disclosure.

FIG. 9 shows two examples of head-mounted display devices according toexamples of the present disclosure.

FIGS. 10A and 10B show a block diagram of a method for displaying visualcontent via a head-mounted display (HMD) device according to examples ofthe present disclosure.

FIG. 11 shows a block diagram of an example computing device accordingto examples of the present disclosure.

DETAILED DESCRIPTION

Head-mounted display (HMD) devices may display visual content to a uservia a virtual reality experience or an augmented reality experience. Forpurposes of the present disclosure, an HMD device provides a virtualreality experience by displaying visual content via an opaque,non-see-through display that creates a fully virtual environment. Forpurposes of the present disclosure, an HMD device provides an augmentedreality experience by displaying visual content via an at leastpartially see-through display that also enables the user to view herreal-world surroundings.

In some examples, virtual or augmented reality HMD devices also may beused to view stereoscopic (stereo) content, such as three-dimensional(3D) movies. Traditionally, stereo content comprises a left eye imageand a right eye image displayed as a stereo pair to provide visual depthcues to a viewer of the stereo content. Such stereo content is typicallycaptured using one or more cameras and is displayed and viewed on afixed-position display, such as a 2D television, 3D television orprojection screen.

Traditional fixed-position stereoscopic displays may not displaystereoscopic content optimally when the user moves his or her head orotherwise changes position. For example, such user movement may resultin stereo visual content displayed at a suboptimal location, which canproduce spatial misperceptions, visual artifacts and/or otherdistortions. For example, shifting and shearing of the displayedimage(s) may occur due to motion of the user's head while viewing stereovisual content via a fixed-position display device, such as a 3Dtelevision.

As described in more detail below, the systems and methods of thepresent disclosure enable HMD devices to modify and display stereoscopicvisual content in a manner that minimizes visual artifacts and otherdistortions created by previous solutions.

FIG. 1 illustrates an example of a computing device 108 communicativelycoupled to an HMD device 104 according to examples of the presentdisclosure. In this example, the computing device 108 is shown as aseparate component from the HMD device 104. The HMD device 104 may becommunicatively coupled to the computing device 108 via a wired orwireless connection. In some examples, the HMD device 104 may becommunicatively coupled to the computing device 108 via a network 156.The network 156 may take the form of a local area network (LAN), widearea network (WAN), wired network, wireless network, personal areanetwork, or a combination thereof, and may include the Internet. Inother examples, the computing device 108 may be integrated into the HMDdevice 104.

The computing device 108 comprises memory 116 holding instructions 120executable by a processor 112 to perform one or more of the methods andprocesses described herein. Additional details regarding the componentsand computing aspects of the computing device 108 are described in moredetail below with reference to FIG. 11.

The instructions 120 may be executed to generate a virtual coordinatesystem 144 including an origin 148 by which visual content 124 may bedisplayed to appear in a fully virtual or augmented reality environment.As described above, the visual content 124 may comprise stereo visualcontent 128, comprising a left eye image 132 and a right eye image 136.As described in more detail below, the instructions 120 also maycomprise one or more scaling factors 152 that may be used to scalevisual content 124.

Each frame of stereo visual content 128 may include a single left eyeimage 132 and a single right eye image 136 to be displayed for theuser's left eye and right eye, respectively. In some examples, the lefteye image 132 and right eye image 136 may be interlaced. In one example,three-dimensional movies may be filmed using stereoscopic cameras thatprovide the perspective of a single camera view for each eye.

In other examples, the visual content 124 may comprise three-dimensionalvisual content 140. Three-dimensional visual content 140 may comprisedata describing a captured object or scene from a plurality ofperspectives. In some examples, three-dimensional visual content 140 maycomprise a photographic recording of a light field that may be displayedvia an HMD device to depict a more complete three-dimensionalrepresentation of the captured subject than available with a singlestereoscopic image or video pair. Examples of three-dimensional visualcontent 140 include holograms.

In some examples, viewing three-dimensional visual content 140 from aplurality of perspectives may be facilitated by generating image datafor the three-dimensional visual content using position and/ororientation data provided by computing device 108 or HMD device 104. Theposition and/or orientation data may be used, for example, to displaythe three-dimensional visual content 140 with a realistic and stableposition and orientation.

Generally when viewing stereo visual content 128, a viewer is limited toviewing the content at the perspective with which the content wascaptured or created. In these situations and as noted above, spatialmisperceptions, visual artifacts and/or other distortions created byuser movement can make it difficult to maintain realism, stability andorientation of the stereo visual content 128. Accordingly, and asdescribed in more detail below, the systems and methods of the presentdisclosure may enable HMD devices to display stereo visual content in amanner that reduces such distortions, enhances realism and provides amore enjoyable viewing experience.

With continued reference to FIG. 1, the HMD device 104 may comprise adisplay 160, a processor 114 and memory 118. In some examples, display160 may be an opaque and non-see through display that provides a virtualreality experience to the user. For example, an HMD device 104 thatprovides a virtual reality experience displays a field of view 164 thatsolely includes virtual content 168. In some examples, images ofreal-world objects 172 from the user's real-world environment may becaptured and used to display corresponding virtual objects within thevirtual environment.

In some examples, the display 160 may comprise an at least partiallytransparent display that presents an augmented reality experiencecomprising a partially virtual three-dimensional environment in whichthe user views her real-world environment along with virtual contentdisplayed to appear within the environment. In some examples, display160 may provide both virtual reality and augmented realityfunctionality. Examples of HMD devices 104 that may display fullyvirtual and partially virtual three-dimensional environments aredescribed in more detail below with respect to FIG. 9.

In some examples, HMD device 104 may display stereo visual content 128by displaying a stereo pair of a left eye image 132 and a right eyeimage 136 to provide stereoscopic depth cues to a user of the HMD device104. In some of the examples illustrated herein, for simplicity, asingle image is illustrated to depict stereo visual content 128 andthree-dimensional visual content 140.

With reference now to the example of FIGS. 2 and 3A, a user 204 may viewvisual content displayed via an HMD device 104 at a variety of locationswithin a room 212. In this example, the HMD device 104 displays stereovisual content 128 in the form of a 3D movie 216 including an evilwizard 218. The HMD device 104 also may display separatethree-dimensional visual content 140 in the form of a holographic cube220. The HMD device 104 displaying the visual content in this example isan augmented-reality HMD device with a field of view 224.

As described above, three-dimensional visual content 140 differs fromstereo visual content 128 for at least the reason that three-dimensionalvisual content may be displayed and viewed from a plurality ofperspectives, with different perspectives revealing new visualinformation that is unavailable from other perspectives. On the otherhand, each image in stereo visual content 128 is captured from a singleperspective of the capture device, thereby limiting the user to viewingsuch images from that perspective.

As illustrated in FIGS. 2 and 3A, in one example both thethree-dimensional cube 220 and the stereoscopic 3D movie 216 may bedisplayed at a default display distance 228 relative to the HMD device104. In this example, the left and right images of the stereoscopic 3Dmovie 216 are displayed floating in the room 216 at a position coplanarwith a front face 232 of the cube 220. The default display distance 228may be a fixed distance that is established to provide the user 204 witha comfortable viewing experience in which the displayed visual contentis neither too close nor too far away from the viewer. FIG. 3Aillustrates a simplified top down view of the room 212 of FIG. 2, inwhich the HMD device 104 is located at a viewing position 304 thatcorresponds to an origin in a three-dimensional virtual coordinatesystem, indicated by the X-Y-Z axes.

In the example of FIGS. 2 and 3A, displaying visual content via the HMDdevice 104 comprises establishing the default display distance 228relative to the origin/viewing position 304 in the virtual coordinatesystem. As shown in FIG. 3A, the default display distance 228 isillustrated extending in the Z-axis direction perpendicularly from anX-Y plane 308 extending through the origin/viewing position 304. Inother examples a default display distance may be established and definedin any other suitable manner.

In the example of FIGS. 2 and 3A, both the stereoscopic 3D movie 216 andthe cube 220 are displayed at the default display distance 228. In someexamples, the default display distance 228 may be between approximately1.25 meters and approximately 5 meters from the origin/viewing position304. In some examples, the default display distance 228 may beapproximately 2 meters from the origin/viewing position 304.

With reference again to the FIGS. 2 and 3A, in some examples the cube220 and/or stereoscopic 3D movie 216 may be displayed in a world-lockedmanner. When displayed in a world-locked manner, visual content appearsto be fixed relative to real world objects and/or other displayedvirtual objects viewable through the HMD device 104. In this manner, awearer of the HMD device may move around a real world physicalenvironment while perceiving the world-locked visual content asremaining stationary in a fixed location and orientation in the physicalenvironment. In the example of FIGS. 2 and 3A, this may allow the cube220 to be displayed and viewed from a plurality of perspectives, withdifferent perspectives revealing new visual information that isunavailable from other perspectives. The HMD 104 may accomplish thisusing a variety of techniques, including head and position tracking asdescribed in more detail below.

In other examples, the HMD device 104 may operate in a body-lock displaymode in which one or more virtual objects may be displayed via the HMDdevice with body-locked positions. When displayed in a body-lockedmanner, a virtual object appears to be fixed relative to the wearer ofthe HMD device 104, and the body-locked position of the virtual objectappears to be moveable relative to real-world objects and other virtualobjects.

As noted above, when a user is viewing stereo visual content via an HMDdevice, user head movement can cause visible distortions in thedisplayed content, such as unnatural shifting and shearing of theimage(s). Such distortions may be particularly prominent when the stereovisual content is displayed in a world-locked manner. The amount ofimage shifting and shearing perceived by a user may depend at least inpart on the distance from the user's head to the displayed image(s). Ingeneral, as the distance from the user's head to the displayed imagedecreases, image shifting/shearing caused by user head movementincreases. For example, stereo visual content displayed and viewed on adesktop computer monitor is more susceptible to shifting and shearingcaused by user head movement as compared to stereo visual contentdisplayed in a movie theater, where the content is displayed at arelatively large distance from the user.

Accordingly, and in one potential advantage of the present disclosure, amodified display distance that is greater than the default distance maybe set, and stereo visual content may be scaled and displayed via an HMDdevice at the modified display distance. In this manner, imagedistortions at the default display distance may be minimized orsubstantially eliminated. Additionally, and as described in more detailbelow, by scaling the stereo visual content by a scaling factorproportional to a difference between the modified display distance andthe default display distance, the content displayed at the modifieddisplay distance may be perceived by the user as being substantially thesame size as if displayed at the default display distance. Thistreatment maintains the angular size of the visual content, so that aviewer who remains still and views one of the stereoscopic images withone eye is not able to discern between original content displayed at thedefault display distance and scaled content displayed at the modifieddisplay distance. The only perceptible difference may arise when theviewer moves his or her head and notes a difference in the degree of ashearing artifact.

With reference now to FIG. 3B, an example of displaying stereo visualcontent at a modified display distance 312 is illustrated. In thisexample, the system may determine that the visual content of the 3Dmovie 216 comprises stereo visual content including left eye images andright eye images. Based on determining that the 3D movie 216 comprisesstereo visual content, the movie may be displayed at the modifieddisplay distance 312.

The modified display distance 312 may be any suitable distance greaterthan the default display distance 228. In different examples, themodified display distance 312 may be approximately 5 meters, 10 meters,20 meters, 30 meters, or other suitable distance. While greater modifieddisplay distances may be desirable, in some examples approximately 10meters may be a suitable display distance for common types of stereovisual content 128. In some examples, different modified displaydistances may be determined and set based on one or more aspects relatedto the stereo visual content, including but not limited to image contentand image quality. Additionally or alternatively, different modifieddisplay distances may be determined and set based on one or more aspectsrelated to the physical space in which the HMD 104 is located, includingbut not limited to dimensions, geometric characteristics, locations ofreal world objects, and surface textures.

As noted above, stereo visual content may comprise a left eye image anda corresponding right eye image. For stereo visual content in the formof a video or movie, each frame of the content may comprise a left eyeimage and corresponding right eye image. Accordingly and in anotheraspect of the present disclosure, in addition to displaying the visualcontent at a modified display distance, each left eye image and righteye image of each frame may be scaled to a scaled left eye image andscaled right eye image, respectively, using a scaling factor 152 that isproportional to a difference between the modified display distance andthe default display distance. In this manner, the apparent size of thestereo visual content at the default display distance may be preservedwhen the scaled stereo visual content is displayed at the modifieddisplay distance.

In some examples, the left eye image 132 and the right eye image 136 maybe scaled by the same scaling factor 152. In other examples, the lefteye image 132 and the right eye image 136 may be treated independently,and each image may be scaled by a different scaling factor 152.

In the example illustrated by FIG. 3B, the scaling factor 152 may beproportional to a difference between the modified display distance 312and the default display distance 228. In one example, the defaultdisplay distance may be 2 m. and the modified display distance may be 4m., such that the modified display distance is twice the default displaydistance. Accordingly, in this example the scaling factor is 2.

With respect to the example shown in FIG. 3A, each frame of the movie216 may have a default width 316 when displayed at the default displaydistance 228. In the present example it follows that, when displayed atthe modified display distance 312, each frame is scaled by the scalingfactor 2 to have a modified width 320 that is twice the default width316, as schematically illustrated in FIG. 3B. In other examples, otherdefault display distances, modified display differences andcorresponding scaling factors may be utilized. Additionally, it will beunderstood that width is one example of a dimension of the stereo visualcontent 128 that may be scaled to provide a more comfortable userexperience. In the present example, each left and right image of eachframe of the movie 216 is rectangular, thus the height of each frame atthe default display distance 228 also is multiplied by the same scalingfactor to preserve the aspect ratio of the movie at the modified displaydistance 312. In another example, the diameter of circular stereo visualcontent 128 may be scaled.

With reference now to the examples shown in FIGS. 4A and 4B and as notedabove, by scaling stereo visual content in this manner, the apparentsize of the content at a default display distance 408 may be preservedwhen displayed at a modified display distance 440. For illustrativepurposes the following description refers to a left eye image 404 of oneframe of stereo visual content. It will be appreciated that thisdescription applies equally to the corresponding right eye image of theframe.

In the example illustrated in FIGS. 4A and 4B, the left eye image 404may be displayed at the default display distance 408 from anorigin/viewing position 404 in a virtual coordinate system as describedabove. An angular size 416 of the left eye image 404 is an angularmeasurement of the size of image 404 as projected on the viewer'sretina. In anatomical terms, the angular size 416 may represent theangle that the eyes of the viewer 412 rotate through between lookingdirectly at a bottom portion 420 of the left eye image 404 to looking ata top portion 424. In the example illustrated in FIG. 4A, the distancebetween the bottom 420 of the left eye image 404 to the top 424 isrepresented by unscaled height 428. Also and for ease of description,the left eye image 404 is depicted in FIG. 4A as it would be seen fromthe perspective of the viewer 412.

FIG. 4B illustrates one example of the left eye image 404 scaled anddisplayed at a modified display distance 440 from the origin/viewingposition 404. In the example illustrated by FIG. 4B, the scaled left eyeimage 404 has a scaled height 436 that is the product of unscaled height428 multiplied by a scale factor as discussed above. In this example,the modified display distance 440 is twice the default display distance408. Accordingly, the scale factor is the modified display distance 440divided by the default display distance 408, yielding a value of 2. Itfollows that the scaled height 436 is twice the unscaled height 428.

By scaling and displaying the left eye image 404 in this manner, the HMDdevice 104 enables the viewer 412 to perceive the scaled left eye image404 at modified display distance 440 as having the same apparent size asthe unscaled left eye image 404 at the default display distance 408.Additionally and as described above, utilizing the modified displaydistance 440 reduces or substantially eliminates certain imagedistortions in stereo visual content, thereby providing the viewer 412with a more pleasant, engrossing and less distracting viewingexperience.

In some examples, the relationship between the dimensions of an item ofvisual content, its display distance and the apparent size of the visualcontent may be described by the following formula, where 0 represents anangular size of the visual content, x represents a dimension of thevisual content, and D represents the display distance:

$\theta = {2\; \tan^{- 1}\frac{x}{2D}}$

With reference to the examples shown in FIG. 4A and 4B, this formuladescribes a relationship in which the angular size 416 of the visualcontent depends on the fraction

$\frac{x}{2D},$

which represents a dimension of the visual content divided by twice thedisplay distance. As a result, the angular or apparent size 416 of thestereo visual content 404 and the scaled stereo visual content 432 mayremain the same to the viewer 412 as the display distance is modifiedand the dimensions of the content correspondingly scaled. In otherexamples, the dimensions of the content may be scaled using any othersuitable approach, and may utilize proportions slightly varying from1:1.

In some examples, a modified display distance may be set based on one ormore physical aspects of the real-world environment of a user. FIG. 5illustrates one example of setting the modified display distance toaccommodate the real-world environment of a viewer 504. The viewer 504is in a room 508 comprising a planar surface in the form of a wall 512that is approximately 5 m. from the viewer. In some examples, utilizinga modified display distance larger than 5 m. may be desirable forgreater reductions of spatial misperceptions such as shearing. However,displaying stereo visual content 216 at a virtual location that is“behind” the wall 512 in the virtual coordinate system may interferewith the perception of viewer 504 in other ways. Accordingly, in thisexample the system may identify the wall 512 as a planar surface in theviewer's real-world environment. Based on this identification, amodified display distance 520 may be set to the distance from the originin the virtual coordinate system to the wall 512. In this example, theorigin may be a point (not shown) on the HMD device 104, and thedimensional line indicating modified display distance 520 extends fromthe Z-axis coordinate of the wall 512 to the Z-axis coordinate of theorigin (indicated by line 530).

Any suitable method may be used to identify planar surfaces. In oneexample, surface reconstruction based on captured images of the room 508may be used to identify planar surfaces and to determine how far thewall 512 is from the HMD device 104 and origin, thus setting themodified display distance 520.

In some examples, surface reconstruction or any other suitable methodalso may be used to determine an amount of texture on a planar surface,such as one or more of contrast, brightness or other characteristics ofthe surface. Such characteristics may be compared to one or morethreshold amounts to determine a modified display distance. In someexamples, heavily textured surfaces may be less desirable as locationsat which stereo visual content is displayed. Accordingly, a thresholdamount of surface texture may be established and utilized to set amodified display distance. For example, if the wall 512 is determined tohave an amount of texture greater than a threshold amount, the modifieddisplay distance may be set to a distance between the wall and theorigin.

With reference again to FIG. 5 and as noted above, the default displaydistance 228 may be utilized to display three-dimensional visualcontent. Accordingly and in some examples, the computing device 108 maydetermine that selected visual content comprises three-dimensionalvisual content. Based on determining that the selected visual contentcomprises three-dimensional visual content, the three-dimensional visualcontent may be displayed at the default display distance. In the exampleof FIG. 5, based on determining that the holographic cube 220 comprisesthree-dimensional visual content, the cube is displayed via the HMD 504at the default display distance 228.

In some examples, a suitable display distance for visual content alsomay be impacted by real-world objects 172 and/or virtual content 168within a real-world or virtual environment. FIG. 6 illustrates oneexample of an occluding object in the form of real-world couch 604 thatmay interfere with the display of stereo visual content at a locationbehind the couch. In this example, the couch 604 is located between theHMD device 104 and the location of stereo visual content 216 at amodified display distance 616.

Like the disadvantages of displaying stereo visual content at a virtuallocation “behind” the wall 512 of FIG. 5, displaying the stereo visualcontent 216 “behind” the couch 604 in FIG. 6 may interfere with theperception of the content by the viewer 620. Accordingly, and in onepotential advantage of the present disclosure, the system may determinethat the couch 604 is located between a location of the HMD device 104in the virtual coordinate system and a location of the stereo visualcontent 216 at the modified display distance 616. Such determination maybe performed, for example, using image data captured by HMD device 104and spatial mapping techniques to establish the locations of the HMDdevice, couch 604 and other real-world and virtual objects within thefield of view of the HMD device.

With reference also to FIG. 7, based on determining that couch 604 islocated between the HMD device 104 and the stereoscopic 3D movie 216 atthe modified display distance 616, the modified display distance may beshortened to a shortened modified display distance 704 that is betweenthe occluding object 604 and the location of the HMD device 608. Usingthe shortened modified display distance 704 and as explained above, thesize of the stereoscopic 3D movie 216 also may be scaled to account forthe shortened modified display distance 704.

In some examples, other virtual content such as a three-dimensionalholographic object may be displayed between the HMD device and stereovirtual content displayed at a modified display distance. In theseexamples, and as with real-world occluding objects discussed above, themodified display distance may be shortened to a shortened modifieddisplay distance that is between the occluding holographic content andthe location of the HMD device.

In some examples, when the system determines that an occluding object islocated between the location of the HMD device 104 in the virtualcoordinate system and a location of visual content at a modified displaydistance, the HMD device may dim at least the occluding object. Anysuitable method or device may be used to dim the occluding object. Inone example, where the HMD device 104 is an augmented reality HMD devicecomprising an at least partially see-through display, the display maycomprise an electrochromatic material that may be selectively tinted toallow less light from a real-world occluding object through the displayto reach the eyes of a viewer. In this manner, dimming may reduce thebrightness of the occluding object and fade out the unwanted real-worldcontent, such as the couch 604 in FIG. 6. As noted below, such dimmingmay be performed locally at the occluding object or, in some examples,globally across the entire display or field of view.

With reference again to FIG. 7, other real-world objects 172 in additionto an occluding object may be present within the field of view 224 of anHMD device 104. In this example, other real-world objects within thefield of view include potted plant 628, bookshelf 632 and coat rack 636.In some examples, all of these other real-world objects 172 within thefield of view 164 of the HMD device 104 may be dimmed. Additionally, insome examples other virtual content, such as the holographic cube 220shown in FIG. 2, may be dimmed.

In some examples, a left eye image and a right eye image of stereovisual content may be modified independently to address one or morevisual issues. Such modifications may include translation, rotation,and/or scaling. In some examples, displaying visual content 124 via anHMD device 104 may comprise determining a misalignment between a lefteye image and right eye image. In some examples, misalignment may occurin stereo visual content 128, such as a 3D movie for example, due topoor capture of the original scene depicted in the content. For example,the misalignment may be due to a first camera capturing a left eye imageat a position that is vertically and/or horizontally misaligned ascompared with the position of a right eye image captured by a secondcamera. Based on determining such misalignment, the displayed positionof at least one of the left eye image and the right eye image may bechanged.

FIGS. 8A and 8B illustrate examples of changing the displayed positionof at least one of a left eye image 808 and a right eye image 812 tocorrect for a vertical misalignment, indicated at 804. The HMD 104 mayaccomplish this using a variety of techniques, including eye tracking asdescribed in more detail below. In this example the left eye image 808is displayed by a left eye display 800L, and the right eye image 812 isdisplayed by a right eye display 800R. In other examples of HMD devices104, a single display system may display both images. After identifyingthe misalignment 804, the displayed position of the left eye image 808may be translated downwardly in the negative y-axis direction towardsthe bottom of the left eye display 800L, as illustrated by dashed line816, until the y-axis positions of the two images are aligned.Additionally or alternatively, the displayed position of the right eyeimage 812 may be translated upwardly in the positive y-axis directiontowards the top of the right eye display 800R, as illustrated by dashedline 820, to align the y-axis positions of the two images.

FIG. 8B illustrates an example of left eye display 800L and right eyedisplay 800R after changing the displayed position of the left eye image808 and the right eye image 812 to correct for the misalignment 804. Insome examples, one or more of cropping and scaling also may be appliedso that the left eye image 808 and the right eye image 812 may be equalsizes.

These and other aspects of the present disclosure may be practiced byHMD devices 104. FIG. 9 illustrates two examples of HMD devices 104. Oneexample of an HMD device 104 is a virtual reality HMD device 904A thatincludes an opaque, non-see-through display 908. Another example of anHMD device 104 is an augmented reality HMD device 904B that comprises anat least partially transparent display 912. It will be appreciated thatthe following descriptions of sensors and systems may apply to both theaugmented reality HMD device 904B and the virtual reality HMD device904A.

In the example of FIG. 9, each of the example HMD devices 904A and 904Bhas a construction that includes a frame 916 that wraps around the headof the user to position a display close to the user's eyes. The frame ofvirtual reality HMD device 904A may include a rigid portion and anelastic portion whereas the frame 916 of augmented reality HMD device904B may be substantially rigid around its circumference. The frame 916may support additional components such as, for example, a processor 920and input devices 924A, 924B, 924C and 924D. The processor 920 includeslogic and associated computer memory 928 configured to provide visualcontent 124 to a display, to receive sensory signals from input devices924A, 924B, 924C and 924D, and to enact various control processesdescribed herein.

Various suitable display technologies and configurations may be used todisplay images via the displays of the HMD devices 904A and 904B. Forexample, in virtual reality HMD device 904A, the display 908 may be anopaque display, such as a non-see-through Light-Emitting Diode (LED)display, a Liquid Crystal Display (LCD), or any other suitable type ofopaque or otherwise non-see-through display. In augmented reality HMDdevice 904B, the display 912 may be an at least partially transparentdisplay that is configured to enable a wearer of the augmented realityHMD device 904B to view physical, real-world objects in the physicalenvironment through one or more partially transparent pixels displayingvirtual object representations. For example, the display 912 may includeimage-producing elements such as, for example, a see-through OrganicLight-Emitting Diode (OLED) display.

As another example of a transparent display, the augmented reality HMDdevice 904B may include a light modulator on an edge of the display 912.In this example, the display 912 may serve as a light guide fordelivering light from the light modulator to the eyes of a wearer. Inother examples, the display 912 may utilize a liquid crystal on silicon(LCOS) display. The display 912 may include both a left L and right Rdisplay in a stereoscopic display configuration. The left L and right Rdisplays each display a view of an augmented reality scene from theperspective of the user's corresponding eye. By viewing the augmentedreality scene through the left L and right R displays, the user willperceive virtual objects as being located at particular depths in thereal world.

The input devices 924A, 924B, 924C and 924D may include various sensorsand related systems to provide information to the processor 920. Suchsensors may include an inertial measurement unit (IMU) 924A, one or moreimage sensors 924B, and one or more ambient light sensors 924C. The oneor more outward facing image sensors 924B may be configured to captureand/or measure physical environment attributes of the physicalenvironment in which the augmented reality HMD device 904B is located.In one example, the one or more image sensors 924B may include avisible-light camera configured to collect a visible-light image of aphysical space. Additionally, the input devices may include a presencesensor 924D that detects whether a user is wearing the HMD device. Inone example, the presence sensor 924D may comprise an inwardly-facingimage sensor configured to determine whether the user's head is adjacentto the sensor, which indicates the user is wearing the HMD device.

In one example of the augmented reality HMD device 904B that includes adisplay 912 having a transparent display type, the position and/ororientation of the augmented reality HMD device 904B relative to thephysical environment may be assessed so that augmented-reality imagesmay be accurately displayed in desired real-world locations with desiredorientations. In both augmented reality HMD device 904B and virtualreality HMD device 904A, the IMU 924A may be configured to provideposition and/or orientation data to the processor 920. The orientationderived from the sensor signals of the IMU may be used to display one ormore holographic images with a realistic and stable position andorientation.

The processor 920 may include a logic processor and the two example HMDdevices 104 may include volatile memory and non-volatile storage, asdiscussed in more detail below with respect to the example computingsystem 1100 of FIG. 11.

FIGS. 10A-10B illustrate a flow chart of a method 1000 for displayingvisual content via a head-mounted display (HMD) device according toexamples of the present disclosure. The following description of method1000 is provided with reference to the software and hardware componentsdescribed herein and shown in FIGS. 1-9 and 11. It will be appreciatedthat method 1000 also may be performed in other contexts using othersuitable hardware and software components.

With reference to FIG. 10A, at 1004, the method 1000 may includeestablishing a default display distance from an origin in a virtualcoordinate system. At 1008, the method 1000 may include setting amodified display distance from the origin. At 1012, the method 1000 mayinclude wherein the modified display distance is approximately 10meters.

At 1016, the method 1000 may include determining that the visual contentcomprises stereo visual content comprising a left eye image and a righteye image. At 1020, the method 1000 may include, based on determiningthat the visual content comprises stereo visual content, scaling theleft eye image to a scaled left eye image and scaling the right eyeimage to a scaled right eye image using a scaling factor that isproportional to a difference between the modified display distance andthe default display distance. At 1024, the method 1000 may includedisplaying the scaled left eye image and the scaled right eye image atthe modified display distance.

At 1028, the method 1000 may include determining that the visual contentcomprises three-dimensional visual content. At 1032, the method 1000 mayinclude displaying the three-dimensional visual content at the defaultdisplay distance.

At 1036, the method 1000 may include determining that an occludingobject is located between a location of the HMD device in the virtualcoordinate system and a location of the visual content at the modifieddisplay distance. At 1040, the method 1000 may include wherein theoccluding object is a real-world object.

With reference now to FIG. 10B, at 1044, the method 1000 may includeshortening the modified display distance to a shortened modified displaydistance that is between the occluding object and the location of theHMD device. At 1048, the method 1000 may include dimming at least theoccluding object. At 1052, the method 1000 may include dimming allreal-world objects and other displayed virtual content within a field ofview of the HMD device. At 1056, the method 1000 may include identifyinga planar surface. At 1060, the method 1000 may include setting themodified display distance to a distance from the origin to the planarsurface. At 1064, the method 1000 may include determining a misalignmentbetween the left eye image and the right eye image. At 1068, the method1000 may include based on determining the misalignment, changing adisplayed position of at least one of the left eye image and the righteye image. At 1072, the method 1000 may include wherein an HMD devicepracticing the method comprises a see-through display.

In some embodiments, the methods and processes described herein may betied to a computing system of one or more computing devices. Inparticular, such methods and processes may be implemented as acomputer-application program or service, an application-programminginterface (API), a library, and/or other computer-program product.

FIG. 11 schematically shows a non-limiting embodiment of a computingsystem 1100 that can enact one or more of the methods and processesdescribed above. Computing system 1100 is shown in simplified form.Computing system 1100 may take the form of one or more gaming consoles,personal computers, server computers, tablet computers,home-entertainment computers, network computing devices, gaming devices,mobile computing devices, mobile communication devices (e.g., smartphones), and/or other computing devices, and wearable computing devicessuch as smart wristwatches and head mounted display devices. In theabove examples, computing device 108 may comprise computing system 1100or one or more aspects of computing system 1100.

Computing system 1100 includes a logic processor 1104, volatile memory1108, and a non-volatile storage device 1112. Computing system 1100 mayoptionally include a display subsystem 1116, input subsystem 1120,communication subsystem 1124, and/or other components not shown in FIG.11.

Logic processor 1104 includes one or more physical devices configured toexecute instructions. For example, the logic processor may be configuredto execute instructions that are part of one or more applications,services, programs, routines, libraries, objects, components, datastructures, or other logical constructs. Such instructions may beimplemented to perform a task, implement a data type, transform thestate of one or more components, achieve a technical effect, orotherwise arrive at a desired result.

The logic processor 1104 may include one or more physical processors(hardware) configured to execute software instructions. Additionally oralternatively, the logic processor may include one or more hardwarelogic circuits or firmware devices configured to executehardware-implemented logic or firmware instructions. Processors of thelogic processor 1104 may be single-core or multi-core, and theinstructions executed thereon may be configured for sequential,parallel, and/or distributed processing. Individual components of thelogic processor optionally may be distributed among two or more separatedevices, which may be remotely located and/or configured for coordinatedprocessing. Aspects of the logic processor may be virtualized andexecuted by remotely accessible, networked computing devices configuredin a cloud-computing configuration. In such a case, these virtualizedaspects are run on different physical logic processors of variousdifferent machines, it will be understood.

Non-volatile storage device 1112 includes one or more physical devicesconfigured to hold instructions executable by the logic processors toimplement the methods and processes described herein. When such methodsand processes are implemented, the state of non-volatile storage device1112 may be transformed—e.g., to hold different data.

Non-volatile storage device 1112 may include physical devices that areremovable and/or built-in. Non-volatile storage device 1112 may includeoptical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.),semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.),and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tapedrive, MRAM, etc.), or other mass storage device technology.Non-volatile storage device 1112 may include nonvolatile, dynamic,static, read/write, read-only, sequential-access, location-addressable,file-addressable, and/or content-addressable devices. It will beappreciated that non-volatile storage device 1112 is configured to holdinstructions even when power is cut to the non-volatile storage device1112.

Volatile memory 1108 may include physical devices that include randomaccess memory. Volatile memory 1108 is typically utilized by logicprocessor 1104 to temporarily store information during processing ofsoftware instructions. It will be appreciated that volatile memory 1108typically does not continue to store instructions when power is cut tothe volatile memory 1108.

Aspects of logic processor 1104, volatile memory 1108, and non-volatilestorage device 1112 may be integrated together into one or morehardware-logic components. Such hardware-logic components may includefield-programmable gate arrays (FPGAs), program- andapplication-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

The terms “program” and “application” may be used to describe an aspectof computing system 1100 typically implemented in software by aprocessor to perform a particular function using portions of volatilememory, which function involves transformative processing that speciallyconfigures the processor to perform the function. Thus, a program orapplication may be instantiated via logic processor 1104 executinginstructions held by non-volatile storage device 1112, using portions ofvolatile memory 1108. It will be understood that different programsand/or applications may be instantiated from the same application,service, code block, object, library, routine, API, function, etc.Likewise, the same program and/or application may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The terms “program” and “application” may encompassindividual or groups of executable files, data files, libraries,drivers, scripts, database records, etc.

It will be appreciated that a “service”, as used herein, is anapplication program executable across multiple user sessions. A servicemay be available to one or more system components, programs, and/orother services. In some implementations, a service may run on one ormore server-computing devices.

When included, display subsystem 1116 may be used to present a visualrepresentation of data held by non-volatile storage device 1112. As theherein described methods and processes change the data held by thenon-volatile storage device, and thus transform the state of thenon-volatile storage device, the state of display subsystem 1116 maylikewise be transformed to visually represent changes in the underlyingdata. Display subsystem 1116 may include one or more display devicesutilizing virtually any type of technology. Such display devices may becombined with logic processor 1104, volatile memory 1108, and/ornon-volatile storage device 1112 in a shared enclosure, or such displaydevices may be peripheral display devices.

When included, input subsystem 1120 may comprise or interface with oneor more user-input devices such as a keyboard, mouse, touch screen, orgame controller. In some embodiments, the input subsystem may compriseor interface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity; and/or any other suitable sensor.

When included, communication subsystem 1124 may be configured tocommunicatively couple various computing devices described herein witheach other, and with other devices. Communication subsystem 1124 mayinclude wired and/or wireless communication devices compatible with oneor more different communication protocols. As non-limiting examples, thecommunication subsystem may be configured for communication via awireless telephone network, or a wired or wireless local- or wide-areanetwork, such as a HDMI over Wi-Fi connection. In some embodiments, thecommunication subsystem may allow computing system 1100 to send and/orreceive messages to and/or from other devices via a network such as theInternet.

The following paragraphs provide additional support for the claims ofthe subject application. One aspect provides, at a computing device, amethod for displaying visual content via a head-mounted display (HMD)device, the method comprising: establishing a default display distancefrom an origin in a virtual coordinate system, setting a modifieddisplay distance from the origin, determining that the visual contentcomprises stereo visual content comprising a left eye image and a righteye image, based on determining that the visual content comprises stereovisual content, scaling the left eye image to a scaled left eye imageand scaling the right eye image to a scaled right eye image using ascaling factor that is proportional to a difference between the modifieddisplay distance and the default display distance, and displaying thescaled left eye image and the scaled right eye image at the modifieddisplay distance.

The method may additionally or alternatively include determining thatthe visual content comprises three-dimensional visual content, and basedon determining that the visual content comprises three-dimensionalvisual content, displaying the three-dimensional visual content at thedefault display distance. The method may additionally or alternativelyinclude, wherein the modified display distance is approximately 10meters.

The method may additionally or alternatively include determining that anoccluding object is located between a location of the HMD device in thevirtual coordinate system and a location of the visual content at themodified display distance, and shortening the modified display distanceto a shortened modified display distance that is between the occludingobject and the location of the HMD device. The method may additionallyor alternatively include, wherein the occluding object is a real-worldobject.

The method may additionally or alternatively include determining that anoccluding object is located between a location of the HMD device in thevirtual coordinate system and a location of the visual content at themodified display distance, and based on determining that the occludingobject is located between the location of the HMD device and thelocation of the visual content, dimming at least the occluding object.The method may additionally or alternatively include dimming allreal-world objects and other displayed virtual content within a field ofview of the HMD device.

The method may additionally or alternatively include identifying aplanar surface, and based on identifying the planar surface, setting themodified display distance to a distance from the origin to the planarsurface. The method may additionally or alternatively includedetermining a misalignment between the left eye image and the right eyeimage, and based on determining the misalignment, changing a displayedposition of at least one of the left eye image and the right eye image.

Another aspect provides a computing device communicatively coupled to ahead-mounted display (HMD) device, the computing device comprising: aprocessor, and a memory holding instructions executable by the processorto establish a default display distance from an origin in a virtualcoordinate system, set a modified display distance from the origin,determine that visual content comprises stereo visual content comprisinga left eye image and a right eye image, based on determining that thevisual content comprises stereo visual content, scale the left eye imageto a scaled left eye image and the right eye image to a scaled right eyeimage using a scaling factor that is proportional to a differencebetween the modified display distance and the default display distance,and cause the HMD device to display the scaled left eye image and thescaled right eye image at the modified display distance.

The computing device may additionally or alternatively include, whereinthe HMD device comprises a see-through display. The computing device mayadditionally or alternatively include, wherein the instructions areexecutable by the processor to: determine that the visual contentcomprises three-dimensional visual content, and based on determiningthat the visual content comprises three-dimensional visual content,display the three-dimensional visual content at the default displaydistance. The computing device may additionally or alternativelyinclude, wherein the modified display distance is approximately 10meters.

The computing device may additionally or alternatively include, whereinthe instructions are executable by the processor to: determine that anoccluding object is located between a location of the HMD device in thevirtual coordinate system and a location of the visual content at themodified display distance, and shorten the modified display distance toa shortened modified display distance that is between the occludingobject and the location of the HMD device. The computing device mayadditionally or alternatively include, wherein the occluding object is areal-world object.

The computing device may additionally or alternatively include, whereinthe instructions are executable by the processor to: determine that anoccluding object is located between a location of the HMD device in thevirtual coordinate system and a location of the visual content at themodified display distance, and based on determining that the occludingobject is located between the location of the HMD device and thelocation of the visual content, dim at least the occluding object. Thecomputing device may additionally or alternatively include, wherein theinstructions are executable by the processor to dim all real-worldobjects and other displayed virtual content within a field of view ofthe HMD device.

The computing device may additionally or alternatively include, whereinthe instructions are executable by the processor to: identify a planarsurface, and based on identifying the planar surface, set the modifieddisplay distance to a distance from the origin to the planar surface.The computing device may additionally or alternatively include, whereinthe instructions are executable by the processor to: determine amisalignment between the left eye image and the right eye image, andbased on determining the misalignment, change a displayed position of atleast one of the left eye image and the right eye image.

Another aspect provides a head-mounted display (HMD) device fordisplaying visual content, comprising: a display, a processor, and amemory holding instructions executable by the processor to establish adefault display distance from an origin in a virtual coordinate system,set a modified display distance from the origin, determine that visualcontent comprises stereo visual content comprising a left eye image anda right eye image, based on determining that the visual contentcomprises stereo visual content, scale the left eye image to a scaledleft eye image and the right eye image to a scaled right eye image usinga scaling factor that is proportional to a difference between themodified display distance and the default display distance, and displaythe scaled left eye image and the scaled right eye image at the modifieddisplay distance.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnon-obvious combinations and sub-combinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. At a computing device, a method for displaying visual content via a head-mounted display (HMD) device, the method comprising: establishing a default display distance from an origin in a virtual coordinate system; setting a modified display distance from the origin; determining that the visual content comprises stereo visual content comprising a left eye image and a right eye image; based on determining that the visual content comprises stereo visual content, scaling the left eye image to a scaled left eye image and scaling the right eye image to a scaled right eye image using a scaling factor that is proportional to a difference between the modified display distance and the default display distance; and displaying the scaled left eye image and the scaled right eye image at the modified display distance.
 2. The method of claim 1, further comprising: determining that the visual content comprises three-dimensional visual content; and based on determining that the visual content comprises three-dimensional visual content, displaying the three-dimensional visual content at the default display distance.
 3. The method of claim 1, wherein the modified display distance is approximately 10 meters.
 4. The method of claim 1, further comprising: determining that an occluding object is located between a location of the HMD device in the virtual coordinate system and a location of the visual content at the modified display distance; and shortening the modified display distance to a shortened modified display distance that is between the occluding object and the location of the HMD device.
 5. The method of claim 4, wherein the occluding object is a real-world object.
 6. The method of claim 1, further comprising: determining that an occluding object is located between a location of the HMD device in the virtual coordinate system and a location of the visual content at the modified display distance; and based on determining that the occluding object is located between the location of the HMD device and the location of the visual content, dimming at least the occluding object.
 7. The method of claim 6, further comprising dimming all real-world objects and other displayed virtual content within a field of view of the HMD device.
 8. The method of claim 1, further comprising: identifying a planar surface; and based on identifying the planar surface, setting the modified display distance to a distance from the origin to the planar surface.
 9. The method of claim 1, further comprising: determining a misalignment between the left eye image and the right eye image; and based on determining the misalignment, changing a displayed position of at least one of the left eye image and the right eye image.
 10. A computing device communicatively coupled to a head-mounted display (HMD) device, the computing device comprising: a processor; and a memory holding instructions executable by the processor to: establish a default display distance from an origin in a virtual coordinate system; set a modified display distance from the origin; determine that visual content comprises stereo visual content comprising a left eye image and a right eye image; based on determining that the visual content comprises stereo visual content, scale the left eye image to a scaled left eye image and the right eye image to a scaled right eye image using a scaling factor that is proportional to a difference between the modified display distance and the default display distance; and cause the HMD device to display the scaled left eye image and the scaled right eye image at the modified display distance.
 11. The computing device of claim 10, wherein the HMD device comprises a see-through display.
 12. The computing device of claim 10, wherein the instructions are executable by the processor to: determine that the visual content comprises three-dimensional visual content; and based on determining that the visual content comprises three-dimensional visual content, display the three-dimensional visual content at the default display distance.
 13. The computing device of claim 10, wherein the modified display distance is approximately 10 meters.
 14. The computing device of claim 10, wherein the instructions are executable by the processor to: determine that an occluding object is located between a location of the HMD device in the virtual coordinate system and a location of the visual content at the modified display distance; and shorten the modified display distance to a shortened modified display distance that is between the occluding object and the location of the HMD device.
 15. The computing device of claim 14, wherein the occluding object is a real-world object.
 16. The computing device of claim 10, wherein the instructions are executable by the processor to: determine that an occluding object is located between a location of the HMD device in the virtual coordinate system and a location of the visual content at the modified display distance; and based on determining that the occluding object is located between the location of the HMD device and the location of the visual content, dim at least the occluding object.
 17. The computing device of claim 16, wherein the instructions are executable by the processor to dim all real-world objects and other displayed virtual content within a field of view of the HMD device.
 18. The computing device of claim 10, wherein the instructions are executable by the processor to: identify a planar surface; and based on identifying the planar surface, set the modified display distance to a distance from the origin to the planar surface.
 19. The computing device of claim 10, wherein the instructions are executable by the processor to: determine a misalignment between the left eye image and the right eye image; and based on determining the misalignment, change a displayed position of at least one of the left eye image and the right eye image.
 20. A head-mounted display (HMD) device for displaying visual content, comprising: a display; a processor; and a memory holding instructions executable by the processor to: establish a default display distance from an origin in a virtual coordinate system; set a modified display distance from the origin; determine that visual content comprises stereo visual content comprising a left eye image and a right eye image; based on determining that the visual content comprises stereo visual content, scale the left eye image to a scaled left eye image and the right eye image to a scaled right eye image using a scaling factor that is proportional to a difference between the modified display distance and the default display distance; and display the scaled left eye image and the scaled right eye image at the modified display distance. 